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For some years now, I have been trying to convince my colleagues in 
structural cognitive developmental psychology that empirical research 
methods might be of use to them. My early professional training was in 
Engineering and Physics, but I found the social sciences to be much more 
interesting, with wide varieties of issues yet to be explored and countless 
unsolved problems. 

My first reaction to reading Piaget was one of incredulity! Here was 
a respected scientist proposing models for problem-solving processes, and 
presenting examples of two or three children, all whose responses confirmed 
his models! Didn't children ever say anything outside his schema? Do all 
children say the same things? Why are many of the terms used so ill-defined 
or circular? Why are experiments described so vaguely that replication is 
near-impossible? 

Now, several years later, with the help of my colleagues and several 
authors in the area, I've begun to understand some of the issues just raised 
and now realize why some of the data analyses T performed for them left them 
just as incredulous about statistics as I was about the "methode clinique." 

Table 1 represents a traditional analysis of data realized in an inter- 
vention designed to provide sufficient cognitive conflict and stimulation 
to increase junior high school students' reasoning ability about moral and 
social issues. The context is a ten week unit in junior high school social 
studies called "Facing History and Ourselves: Holocaust and Human Behavior, 
authored by Margot Strom and Bill Parsons in Brookline, Massachusetts. 
There were experir ^t^^ and control classes in two schools of differing 
social class and achievement. However, pretest measures and reading achieve 
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ment levels were nearly equal for combined experimental and control groups. 

The analysis shows a statistically significant difference in level of 

reasoning about social dilemmas even after initial differences in reading 

level and pretest scores were controlled for. Happily, the difference 

favors the experimental group. 

Today I would like to discuss the implications of such an analysis 

and offer some suggestions about alternative analyses readily available to. 

but often ignored by researchers in this area. 

The first issue concerns the importance of testing the difference 
between two means, the average reasoning levels of the experimental and 
control groups. We might ask. "What is interesting about the average way 
thtrty children solve a social or moral dilemma?" Or. we might be quite 
sophisticated in statistics. and be concerned that the data is not continuous 
and is at best, ordinal, so that the arithmetic mean is an inappropriate 

measure of central tendency. 

The second issue is concerned with describing change in subjects over 
time. What is required is a way of analyzing the stage responses in two 
groups at two times in a way that preserves the integrity of the data, yet 
Hows us to determine the likelihood that any differences may be due to 
hance or random error. I would like to propose a well known, but seldom 
employed non-parametric analysis which deals with the two issues at hand: 
an alternative to group means and a repeated measure design on a variable 

which is ordinal at best. 

Since subjects- scores are often recorded as a stage of reasoning or 
a transitional state between two stages, it is relatively easy to construct 
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a frequency distribution of subjects across stages. Then, if cumulative 
distributions for control and experimental groups are computed, they may 
be easily compared using the Kolmogorov-Smirnov two sample test. This 
statistic tests the null hypothesis that two samples have been drawn from 
the same population by comparing their whole distributions, rather than 
specific parameters and making no assumptions about the shapes. This com- 
parison allows us to test the assumption that the groups are similar on 
the pretest. 

If we are satisfied that the groups were similar on the pretest, we 
may choose to perform the same test on the post-test scores. Or, if we 
wish to find out whether the experimental group changed more than the 
control group, we may classify the subjects by the amount of change in 
their scores, considering the change from transition into consolidation, 
or vice versa, as a one-half stage movement. Then, once again, a frequency 
distribution of change scores may be calculated for both groups and com- 
pared using the two sample Kolmogorov-Smirnov test. 

Table 2 represents the raw data for both experimental and control 
groups on the pretest (Time 1) and the posttest (Time 2). From these 
cross-tabulations it is possible to see who changed and how much. 

Tables 2A and 2B illustrate the frequency distribution of changes in 
stage scores for Loevinger ego development, and issues within Kohlberg's 
system of stages of moral reasoning and Selman's stages of interpersonal 
awareness. The number of regressions is within the range of chance, as 
might be expected, given the interrater reliability of about 0.8. 
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A more coarse analysis is represented in Table 3. Here, the data 
are collapsed to compare subjects who advance or regress over the length 
of the program. 

Table 4 shows the results of applying a more stringent criteria for 
change. Rather than merely consolidating reasoning already available to 
a subject in new settings, change here is defined as .showing reasoning 
characteristic of the next highest stage, not previously elicited. 

In these two cases, there are only two outcomes, for example, 
advancing or not, or exhibiting higher stage thinking or not. Here, the 
binomial test using the proportion attained by the control group as the 
P parameter and testing for the likelihood of the experimental group's 
proportion arising due to chance, serves as a test for comparing the 
two groups. 

More elaborate analyses, beyond the scope of this paper, hut presently 
under investigation, are concerned with change in subjects in several 
applications areas. Noting changes in the patterns of scores may allow 
alternative possibilities for empirically determining decolage. Then we 
can study change in even more complex senses and observe the presently ill- 
defined phenomena of emergence and consolidation. 

The non-parametric tests have been available for many years, but the 
reluctance of structural developmental ists to employ any statistical analy- 
sis has prevented these tools from being used. Hopefully, this paper will 
show the v-elevance and utility of such techniques and aid the developmen- 
talists in establishing their research on a more traditional scientific 
basis. (j 
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Table 1 
Traditional Analysis 







Experimental 


Control 




N 


M S 


N 


M 


S 


Reading Level (1) 


42 


9.35 2.24 


21 


9.84 


2.46 


Interpersonal Awareness (1) 


42 


240.4 34.6 


21 


244.4 


30.2 


Interpersonal Awareness (2) 


42 


255.4 31.4 


21 


241.0 


36.1 



********** 



Analysis of Covariance 



Source of Variation 



Expertmentcl vs. Control 4,499,438 
Within groups 35,081.234 



1 

59 



Total 



39,580.672 



60 



Degrees of Mean 
Sum of Squares Freedom Square 



4,499.438 7.567* 
594.597 



*p = .008 
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Table 2 
Pnoporticns of Subjects at 
Each Stage of Interpersonal Awareness 



Experimental Group 


1(2) 


Time 

2 


2(3) 


3 


• 


1(2) 




.00 


.02 


.00 




9 
C 




,^^^09^ 


^20 


.02 


Time 1 








^^Jl\ 


^09 


2(3) 


.00 






3 


.00 


.00 


.04^ 










Time 


2 




Control Group 




1(2) 


2 


2(3) 


3 




2 


.00 


^29^ 


^10 


.00 


Time 1 


?(3) 


.05 


.10^> 


^28^ 


<>^09 




3 


.00 


. .00 


.05 


^>^05 
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Table 2A 

Proportion of Subject? Changing Their Stage Score 



EXPERIMENTAL 

1 


■ • CONTROL 






-1 


"1/2 


0 


+1/2 


+1 


+1 1/2 


H 


-1 


-1/2 


0 


+1/2 


+1 


+1 1/2 


N 


Personality 


IC 




.25 


.58 


.17 






12 




.44 












Personality 
Change 


ID 


.08 


.11 


.46 


.19 


.15 




26 




.27 


•33 


.33 


.0/ 




10 


Formation 


2A 


.05 


.24 


.43 


.14 


,11 


.03 


37 




.38 


.38 


.19 


.05 




lb 


Closeness/ 
Intiniacy 


2B 




.17 


.50 


.29 


.04 




24 




.38 


.31 


.23 


.08 




\i 


Group Cohesion 


2C 


.03 


,06 


.41 


.41 


.09 




32 


' .19 


.00 


.38 


,25 


.19, 




16 


Trust/ 

Responsibility 


2D 


.03 


.12 


.41 


.44 






34 




.31 


.38 


.31 






lo 


Conformity 


2E 


.03 


.10 


.59 


.24 


.03 




29 


,07 


.14 


.50 


.21 


.07 




14 


Rule-Orientation 2F 


.05 


,10 


.38 


.24 


.19 


.05 


21 




,do 


.62 

■ 


.15 






13 


Decision-Making 2G 
Organization 


.03 


.12 


.48 


.24 


.12 




33 


.06 


.26 


.50 


.13 


.06 




16 


Leadership 


2H 




no 




.00 










?8 


.57 


.14 






7 


Jealousy/ 
Esclusion 


21 




.40 


.20 


.20 


.20 




5 




.33 


.67 








3 


Conflict 
Resolution 


2J 




.23 


.33 


.29 


.12 


.03 


34 




.25 


.65 


.05 


.00 


.05 


20 


Termination 


2K 


.07 




.2c 


.44 


.15 




27 


.08 


.17 


.50 


.17 


.08 
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.56 


.29 


.04 




45 i 


.05 


.14 


.62 


.19 






21 
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Table 2B 

Proportion of Subjects Changing Their Stage Score 



CONTROL 





-] 


-1/2 


t if A 


04 


.38 


Law 




.10 


Morality/ 
Conscience 




.25 


Punishment ■ 


,04 


.04 


Contracts 




.05 


Authority 







, All 0 .1/2 .1 ^1 1/2 N .1 1/2 -1 -1/2 0 .1/2 .1 .1 1/2 ^2 N 



.25 .61 .14 



„ll .04 



.93 .07 



.11 .67 .22 



26 



.10 .43 .30 .10 .07, 30 



28 

26 
21 
14 



37 



,10 .10 .60 .20 

.18 ,18 .18 .18 .09 .09 

.22 .22 .44 .11 

.10 .10 .10 .40 .30 

.11 .55 .33 
1.00 

■ .13 .13 .47 .20 .07 



10 

11 



15 



,12 .56 .16 .06 



50 



.03 ■ .53 .30 .13 
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Table 3 

Proportion of Students Who Change Their Stage Score 







+ 




Control 


Interpersonal Ax^7areness 




- 


+ 


Personality 


.25 


.17 


.4^ 


.22 


Personality change 


.19 


.34 


.27 


.40 


Formation 


.29 


.28 


.38 


.25 


Closeness/Intimacy 


.17 


.33 


.38 


.31 


Group Cohesion 


.09 


.50 


.19 


.44 


Trust/Reciprocity 


.15 


.4^ 


.31 


.31 


Confonnity 


.13 


.27 


.21 


.28 


Role Orientation 


.15 


.48 


.23 


.15 


Dec. Making/Org. 


.15 


.JO 


. 


19 


Leadership 


.16 


.33 


.28 


.14 


Jealousy/Exclusion 


.40 


.40 






Conflict Resolution 


.23 


.44 


.25 


.10 


Termination 


.14 


.59 


.25 


.25 


Global 


.11 






.19 


M:)ral Judgment 










Life 


.42 


.iz 




20 




.10 


.14 


.18 


.54 




.25 


.14 


.4i 


.11 


Jbrality/Conscience 


.31 


.30 


.30 


Punishment 


.08 


.28 


.11. 


.33 


Contracts 


.05 


.07 


.00 


.00 


Authority 


.00 


.22 


.26 


.27 


Global 


.11 








Ego Development 










Ego * " . - 


.12 


-:22 


i:D3 


;43 



Table 4 

Proportion of Subjects Showing Reasoning at a New Stage 





Exp. 


Control 


Personality 


.00 


.00 


Personality Change 


.15 


.07 


Formation 


.14 


.25 


Closeness /Intimacy 


.04 


.08 


Group Cohension 


.31 


.25 


Trus t/Eeciproci ty 


.18 


.25 


Conforndty 


.10 


.21 


Eule Orientation 


.H-J 


08 


Dec. Making/Org. 


.30 


.06 


Leadership 


.25 


.00 


Jealousy/Exlusion 


.20 


.00 


Conflict Resolution. 


.29 


.10 


Termination 


.37 


.25 


Global 


.24 


.10 


Life 




.00 ' 




.30 


.55 


Morality /Conscience 


.07 


.11 


Punishment 


.20 




Ccntracts 


.05 


.11 


Authority 


.00 


.00 


Global 


.03 


.13 


Ego 


.16 


.20 - 
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